Simulation study of learning automata games in automated highway systems

We propose an artijicial intelligence technique called stochastic learning automata to design an intelligent vehicle path controller. Using the information obtained by on-board sensors and local communication modules, two automata are capable of learning the best possible actions to avoid collisions. Although the learning approach taken is capable of providing a safe decision, optimization of the overall traf lc j low is required. This can be achieved by studying the interaction of the vehicles. The design of the adaptive vehicle path planner based on local information is extended with additional daision structures by analyzing the situations of conflicting desired vehicle paths. The analysis of the situations and the design of these structures are made possible by treatment of the interacting reward-penalty mechanisms in individual vehicles as automata games. 1 . INTRODUCTION One of today’s most serious social, economical, and environmental problems is the traffic congestion. To increase highway safety while reducing congestion, US Department of Transportation has taken an approach called the Intelligent Transportation Systems (ITS). A major element of ITS development effort is the Automated Highway Systems (AHS). Vehicle control is probably the most important part of the advanced AHS applications, because technological requirements of such a system are well beyond human capabilities. A large group of investigators is working on vehicle control issues [2]. However, being able to control vehicle dynamics does not necessarily mean that we have an AHS. In an environment with many fast-moving vehicles, making the right decision to avoid collisions and optimize the vehicle path is difficult. Initial research on automated vehicle control indicates that a planning system that can guarantee optimal operation with a sound theoretical background has not yet been developed, and it may be vital to AHS implementation [2]. We visualize two learning automata employing a reinforcement learning algorithm as the heart of our path planner. Using local sensor and limited communications data, the automata learn the optimal actions to be taken for a given situation. Given enough time and correct learning parameters, the automata indicate the best actions to take, and send these actions to the lower control layer. The initial decision system uses mainly local information, and consequently, the actions leamed by the intelligent controller are not globally optimal; the vehicles can survive, but may not be able to reach some of their goals. To overcome this problem, we treat pairs of automata as interconnected automata structures and visualize the interaction between vehicles as sequences of games played between automata. By evaluating these games, it is possible to design new decision rules, and to analyze the interactions between vehicles. 2. A LEARNING METHOD FOR NAVIGATION Recent research on intelligent vehicle includes adaptive intelligent vehicle modules designed to answer the need for real-time maneuver selection for tactical driving [5]. Another approach to intelligent control for autonomous navigation uses a decision-theoretic approach with probabilistic networks where the problem is modeled as partially observable Markov process, and the optimal action is a function of the current belief state [I] . Similarly, a rule-based navigation system that uses worstcase decision-making is defined in [4]. Our approach differs from the above-mentioned works in the use of learning paradigm. Instead of learning the parameters affecting the firing of actions on repeated runs, the automata learn which action to fire based on the local sensor information. In other words, the higher level leaming is not in the design phase, but in the run phase. There are no “prescribed conditions” for actions. The idea of defining a “fixed” structure to be utilized to find the optimal action has its own appeal, since the performance of the system is deterministic in the sense that the best action for a specific situation is known. However, drivers do not follow rules deterministically. In this sense, the learning automata approach is able to capture the dynamics of driver behavior. A crucial advantage of learning compared to other learning approaches is that it requires no information about the environment except for the reinforcement signal. The learning paradigm of the stochastic automaton is based on repeated actions and the resulting 936 0-7803-4269-0/97/S10.00